Extending Item Response Theory to Online Homework

Item Response Theory becomes an increasingly important tool when analyzing ``Big Data'' gathered from online educational venues. However, the mechanism was originally developed in traditional exam settings, and several of its assumptions are infringed upon when deployed in the online realm. For a large enrollment physics course for scientists and engineers, the study compares outcomes from IRT analyses of exam and homework data, and then proceeds to investigate the effects of each confounding factor introduced in the online realm. It is found that IRT yields the correct trends for learner ability and meaningful item parameters, yet overall agreement with exam data is moderate. It is also found that learner ability and item discrimination is over wide ranges robust with respect to model assumptions and introduced noise, less so than item difficulty

Using artificial-intelligence tools to make LaTeX content accessible to blind readers

Screen-reader software enables blind users to access large segments of electronic content, particularly if accessibility standards are followed. Unfortunately, this is not true for much of the content written in physics, mathematics, and other STEM-disciplines, due to the strong reliance on mathematical symbols and expressions, which screen-reader software generally fails to process correctly. A large portion of such content is based on source documents written in LaTeX, which are rendered to PDF or HTML for online distribution. Unfortunately, the resulting PDF documents are essentially inaccessible, and the HTML documents greatly vary in accessibility, since their rendering using standard tools is cumbersome at best. The paper explores the possibility of generating standards-compliant, accessible HTML from LaTeX sources using Large Language Models. It is found that the resulting documents are highly accessible, with possible complications occurring when the artificial intelligence tool starts to interpret the content

Performance of the Pre-Trained Large Language Model GPT-4 on Automated Short Answer Grading

Automated Short Answer Grading (ASAG) has been an active area of machine-learning research for over a decade. It promises to let educators grade and give feedback on free-form responses in large-enrollment courses in spite of limited availability of human graders. Over the years, carefully trained models have achieved increasingly higher levels of performance. More recently, pre-trained Large Language Models (LLMs) emerged as a commodity, and an intriguing question is how a general-purpose tool without additional training compares to specialized models. We studied the performance of GPT-4 on the standard benchmark 2-way and 3-way datasets SciEntsBank and Beetle, where in addition to the standard task of grading the alignment of the student answer with a reference answer, we also investigated withholding the reference answer. We found that overall, the performance of the pre-trained general-purpose GPT-4 LLM is comparable to hand-engineered models, but worse than pre-trained LLMs that had specialized training

"Openness": Weniger ist mehr?

Open Educational Resources führen im Bereich der regulären Hochschulausbildung ein Schattendasein, welches bei weitem nicht ihrem Potential Genüge tut. Der Artikel bespricht einige mögliche Adoptionshürden und zeigt auf, wie ein modifiziertes Verständnis von „Openness“ in Verbindung mit einer dementsprechenden Plattform diese Hürden ausräumen kann. Anhand von Nutzungsdaten aus einem etablierten System wird dargestellt, wie eine solche Architektur zu einem nachhaltigen Geben und Nehmen von Bildungsressourcen unter Lehrenden führen und obendrein innovative Lehrmodelle unterstützen kann. 04.11.2013 | Stefan Dröschler (Wolfenbüttel), Gerd Kortemeyer (East Lansing) & Peter Riegler (Wolfenbüttel

A canvas for the ethical design of learning experiences with digital tools

The use of digital tools has drastically increased in engineering education, accelerated by the COVID-19 pandemic. These tools generate important ethical issues, in particular in terms of privacy and fairness. However, very few teacher training programmes address those topics, which means that teachers are often left to figure out by themselves how to address these issues when they want (or have) to use digital tools in their teaching. In this workshop, participants will be introduced to a pragmatic approach to the ethical design of learning experiences that involve digital tools using a visual thinking guide called a ‘canvas’. Applied and hands-on, this workshop will help participants to develop a practical understanding of the specific ethical issues related to the use of digital tools in teaching and to integrate ethical reflection into design processes when digital technology is involved

Analyzing the impact of course structure on electronic textbook use in blended introductory physics courses

We investigate how elements of course structure (i.e., the frequency of assessments as well as the sequencing and weight of course resources) influence the usage patterns of electronic textbooks (e-texts) in introductory physics courses. Specifically, we analyze the access logs of courses at Michigan State University and the Massachusetts Institute of Technology, each of which deploy e-texts as primary or secondary texts in combination with different formative assessments (e.g., embedded reading questions) and different summative assessment (exam) schedules. As such studies are frequently marred by arguments over what constitutes a “meaningful” interaction with a particular page (usually judged by how long the page remains on the screen), we consider a set of different definitions of “meaningful” interactions. We find that course structure has a strong influence on how much of the e-texts students actually read, and when they do so. In particular, courses that deviate strongly from traditional structures, most notably by more frequent exams, show consistently high usage of the materials with far less “cramming” before exams.National Science Foundation (U.S.) (Grant DUE-1044294)Google (Firm